Flight Simulator Research Articles

Trajectory tracking control of parallel manipulator actuated with shape memory wire

This study describes the design of a parallel spatial manipulator with four degrees of freedom actuated with shape memory alloy (SMA) wire to validate the use of SMA in complicated mechatronics systems. The manipulator has a closed kinematic structure, which includes a fixed base and a moving square platform (end effector). The four arms of the manipulator are SMA wires fastened between the fixed base and the end effector. SMA wire-based actuators replace bulky conventional revolute actuators. This work spotlights the development of an actuator model, dimensional analysis, and design of cascade control strategies of various PID and sliding mode controller in their integral and fractional order configurations. Experimental evaluation of the actuator is performed through trajectory tracking to quantify the different controller configurations. The experimental results indicate that the parallel manipulator associated with SMA wire actuators is the best alternative to conventional motion stages for highly precise micro-positioning and tracking applications in the fields of 3D printing, intricate surgical operations, the medical and pharmaceutical industries, and flight and gaming simulators.

Triggering Cockpit Alerts Using an Eye-Tracking-Based Measure of Monitoring Performance

Abstract: This study explored the potential for enhancing pilot performance via an alerting system that adapts according to an eye-tracking-based measure of monitoring. The novel measure combines gaze analysis with system state assessment to estimate the pilot’s understanding of the current system state. On this basis, an alerting system was developed to direct pilot attention to unnoticed system state changes, thereby improving system state monitoring. In a flight simulator study involving 10 participants in a generic jet cockpit, we compared the adaptive alerting system with a no-assistance condition. Although alerting improved the participants’ performance in two tracking tasks, it adversely impacted performance in a third task. Nonetheless, alerting resulted in a decrease in both variance and mean detection time of critical changes. Participants’ subjective ratings were generally positive, yet they criticized the lack of transparency of the alerting mechanism. Alerts triggered based on eye-tracking and system state show potential for improving operator task performance. Nonetheless, for the system to reach its full performance potential, it is critical that the operator understand the principles underlying the alert triggers. False positives and alert design were identified as key areas for improvement to maintain user trust and task flow.

Basic Computational Algorithms for Representing an Aircraft Flight (Calculation of 3D Displacement and Displaying)

This manuscript describes the computational process to calculate an airplane path and display it in a 2D and 3D coordinate system on a computer screen. The airplane movement is calculated as a function of its dynamic’s conditions according to physical and logical theory. Here, the flight is divided into maneuvers and the aircraft conditions are defined as boundary conditions. Then the aircraft position is calculated using nested loops, which execute the calculation procedure at every step time (Δt). The calculation of the aircraft displacement is obtained as a function of the aircraft speed and heading angles. The simulator was created using the C++ programming language, and each part of the algorithm was compiled independently to reduce the source code, allow easy modification, and improve the programming efficiency. Aerial navigation involves very complex phenomena to be considered for an appropriate representation; moreover, in this manuscript, the influence of the mathematical approach to properly represent the aircraft flight is described in detail. The flight simulator was successfully tested by simulating some basic theoretical flights with different maneuvers, which include stationary position, running along the way, take off, and some movements in the airspace. The maximum aircraft speed tested was 120 km/h, the maximum maneuver time was 12 min, and the space for simulation was assumed to be without obstacles. Here, the geometrical description of path and speed is analyzed according to the symmetric and asymmetric results. Finally, an analysis was conducted to evaluate the approach of the numerical methods used; after that, it was possible to confirm that precision increased as the step time was reduced. According to this analysis, no more than 500 steps are required for a good approach in the calculation of the aircraft displacement.

IMMERSIVE ANALYTICS

Immersive Analytics focuses on the benefits and challenges of using immersive environments for data analysis, and developing designs to improve efficiency. Although immersive technologies are widely available, practical solutions have not gained widespread acceptance in real- world applications. Research in this field focuses on abstract 3D visualization, immersive environments, paper sampling and use case evaluation. 03 Related Works • Brooks early review of VR applications found it effective in specific domains like flight simulators, automotive engineering, and astronaut training. • Van Dam et al. highlighted VR applications for SciVis, benefiting archaeology and medical fields. • Laha and Bowman reviewed VR techniques for visualizing volume data, highlighting the need for controlled experiments to explore individual components of immersion. • Reda et al. summarized research for hybrid reality environments like the CAVE2, emphasizing the possibility of collaborative data analysis. • Brath collected evidence that 3D visualizations offer advantages beyond 2D, focusing on immersive displays. 05 Result • Overall, immersive analytics can lead to better data comprehension, better decision-making, more engagement and teamwork, effective big data exploration, creative data visualization methods, non-technical users' empowerment, and applications in a variety of fields. • These results add to the increasing importance of immersive analytics as an efficient tool for decision support and data analysis. Introduction • To facilitate data-driven analytical reasoning, immersive analytics leverages interactive technology like as virtual reality glasses, big flat screen displays, and even the internet of things. • Immersion refers to an experience that creates a genuine sense of presence in a virtual environment. • A person experiences a shift in awareness from their immediate real environment to another reality. • Immersion analytics is still a relatively new field that has mostly been studied in use cases and workshops. • The notion remains extremely difficult, even within the small community of immersive analytics practitioners. Methodology 4.1 Immersive environments • The study focuses on immersive environments leading to a mixed reality experience. • Abstract 3D visualizations must be presented in a mixed or VR environment where hardware and user interact closely. 4.2 Abstract 3D visualizations • Data that lacks a physical representation or intrinsic spatial organization is referred to as abstract data. • In visualization, abstraction is obtained by using colors and shapes that are not directly associated with the object. 4.3 Paper sampling • Paper sampling in immersive analytics refers to the process of selecting and gathering relevant research papers and publications related to immersive analytics. Conclusion and future scope • In conclusion, immersive analytics has the potential to completely transform how we work with data by facilitating deeper understanding, better decision-making, and improved teamwork. • Immersion analytics will develop further and contribute significantly to data analysis and decision support in the future by tackling issues and seizing opportunities in technological innovation, domain-specific applications, ethical issues, and user experience design.

Validating Synthetic Data for Perception in Autonomous Airport Navigation Tasks

Autonomous navigation within airport environments presents significant challenges, mostly due to the scarcity of accessible and labeled data for training autonomous systems. This study introduces an innovative approach to assess the performance of vision-based models trained on synthetic datasets, with the goal of determining whether simulated data can train and validate navigation operations in complex airport environments. The methodology includes a comparative analysis employing image processing techniques and object detection algorithms. A comparative analysis of two different datasets was conducted: a synthetic dataset that mirrors real airport scenarios, generated using the Microsoft Flight Simulator 2020®video game, and a real-world dataset. The results indicate that models trained on a combination of both real and synthetic images perform much better in terms of adaptability and accuracy compared to those trained only on one type of dataset. This analysis makes a significant contribution to the field of autonomous airport navigation and offers a cost-effective and practical solution to overcome the challenges of dataset acquisition and algorithm validation. It is thus believed that this study lays the groundwork for future advancements in the field.

Numerical investigation of nitrogen spontaneous condensation over airfoils in cryogenic wind tunnel

Cryogenic wind tunnels (CWT) provide the aerodynamic tests that occur at high Reynolds numbers by operating under cryogenic environment to meet the requirements of hypersonic flight simulation. However, condensation in cryogenic environment near the limited condition destroys the reliability of the test measurement results. Accordingly, research on condensation in CWT is imperative for devising effective control strategies. This study introduces a transonic non-equilibrium condensation model, tailored to evaluate the complex phenomena of nitrogen condensation over airfoils. The model was applied to assess condensation effects on the NACA 0012–64 airfoil and the RAE 2822 airfoil under various operating conditions. Furthermore, the study examined the effect of the angle of attack and airfoil shape on the condensation process. The result show that nitrogen condensation near the airfoil is observable only under operating conditions within the pre-condensation zone. Operating conditions of CWT can be tailored to prevent condensation with specific airfoil profiles. The effect of condensation on the pressure coefficient includes advancing the position of the pressure jump and increasing the pressure coefficient of the airfoil intermediate region, particularly within the 20 %–70 % chord length span on the airfoil's upper surface. This primarily affects the measurement of lift coefficients in CWT. In the operating conditions of this study, the lift coefficient exhibits a maximum deviation of 31.4 %, and the drag coefficient shows a maximum deviation of 2.4 %. The transonic spontaneous condensation model offers an approach to evaluate condensation's impact on airfoil tests within CWT. The findings introduce a methodological framework for effectively controlling condensation phenomena.

Wind Shear Operation-Based Competency Assessment Model for Civil Aviation Pilots

Undesirable meteorological conditions are typical aviation flight safety threats. Although most meteorological radar and flight augmentation computers have avionic system wind shear alarms, the preferred approach is that pilots avoid unsafe wind shear events. Therefore, effective pilot competency evaluations are needed to assess pilots’ abilities to deal with these events. This study developed a wind shear operation competency model that includes observable behavior indicators, sub-task decomposition, and competency check items. An adapted competency model and a quantitative data-driven competency evaluation criteria optimization method were then developed using three-dimensional competency feature modeling, after which wind shear simulation flight training data were used to verify the effectiveness of the proposed method. The competency assessment grades were significantly correlated with the results from experienced examiners with a 93.33% evaluation accuracy. The proposed quantitative data-driven competency assessment method can provide effective pilot competency assessments for a range of aviation meteorological threats.

The effect of Ashtanga-Vinyasa Yoga method on air force pilots’ operational performance

IntroductionIn today’s military landscape, optimizing performance and bolstering physical health and mental resilience are critical objectives. Introducing a 12-week Ashtanga Vinyasa Yoga Supta Method (AVYSM) to the training protocol of military trained Airforce pilots, we aim to assesses the feasibility and impact of the method.Materials and equipmentBorg Scale assesses the intensity level of physical activity during the intervention. Flight simulator data gauges operational performance responses. Postural control responses are measured using a force platform, stress responses are monitored via heart monitor, and handgrip dynamometry will measure strength. Respiratory capacity is assessed using a spirometer, body composition is evaluated using impedance balance, and aviation-related questionnaires are administered before and after the intervention period.MethodsIn a randomized controlled trial, the totality of pilots from the “Masters in Military Aeronautics: aviator pilot specialist” course at the Portuguese Air Force Academy (PAA) were randomly assigned to the yoga intervention or the waiting list control groups, with participants providing written informed consent. The control group followed protocolized course classes for 12 weeks, while the intervention group integrated two weekly one-hour yoga sessions into their course.ResultsThe PAA has approved the implementation of this intervention protocol at Airbase 11 in Beja, highlighting its significance for the organization’s policy makers. We hypothesize that this method will enhance operational performance and, subsequently, elevate flight safety.DiscussionThis research’s potential extends beyond the PAA, as it can be adapted for use in Airforce departments of other nations and various military contexts.Clinical trial registrationÉvora University research ethics committee—approval number 21050. Study registered on ClinicalTrials.gov under identifier NCT05821270, registered on April 19, 2023.

Aerodynamic Modeling of a Missile Combining Transfer Learning and Dendritic Net

During the initial stages of missile design, finding an efficient method for analyzing aerodynamic characteristics is crucial. This paper proposes a novel aerodynamic modeling method based on a limited computational fluid dynamics (CFD) dataset, combining transfer learning (TL) with Dendritic Net (DD). Initially, we employ CFD-calculated aerodynamic data to establish a pretrained model using DD. Subsequently, the model is adapted to the target domain by TL, predicting aerodynamic parameters under specific conditions. The overall aerodynamic parameters are utilized to generate a relational spectrum through DD’s white-box features, from which primary features are extracted to establish an aerodynamic polynomial model. Finally, the model’s practicality is validated by ballistic flight simulations. The innovation lies in leveraging DD to generate a relational spectrum of aerodynamic parameters, leading to a high-precision polynomial model. Research shows DD outperforms traditional cell-based networks in predicting aerodynamic parameters, and TL reduces the CFD computation workload in the target domain by 3/4 while maintaining prediction accuracy. The polynomial model exhibits superior accuracy compared to the empirical fitting formulas. The method reduces the computational workload for aerodynamic data collection, and through system identification, a high-precision polynomial model is obtained, which provides a reliable basis for missile controller design.

Computational Tool for Aircraft Fuel System Analysis

Fuel level gauging in aircraft presents a significant flight mechanics challenge due to the influence of aircraft movements on measurements. Moreover, it constitutes a multidimensional problem where various sensors distributed within the tank must converge to yield a precise and single measurement, independent of the aircraft’s attitude. Furthermore, fuel distribution across multiple tanks of irregular geometries complicates the readings even further. These issues critically impact safety and economy, as gauging errors may compromise flight security and lead to carrying excess weight. In response to these challenges, this research introduces a multi-stage project in aircraft fuel gauging systems, as a continuum of studies, where this first article presents a computational tool designed to simulate aircraft fuel sensor data readings as a function of fuel level, fuel tank geometry, sensor location, and aircraft attitude. Developed in an open-source environment, the tool aims to support the statistical inference required for accurate modeling in which synthetic data generation becomes a crucial component. A discretization procedure accurately maps fuel tank geometries and their mass properties. The tool, then, intersects these geometries with fuel-level planes and calculates each new volume. It integrates descriptive geometry to intersect these fuel planes with representative capacitive level-sensing probes and computes the sensor readings for the simulated flight conditions. The method is validated against geometries with analytical solutions. This process yields detailed fuel measurement responses for each sensor inside the tank, and for different analyzed fuel levels, providing insights into the sensors’ signals’ non-linear behavior at each analyzed aircraft attitude. The non-linear behavior is also influenced by the sensor saturation readings at 0 when above the fuel level and at 1 when submerged. The synthetic fuel sensor readings lay the baseline for a better understanding on how to compute the true fuel level from multiple sensor readings, and ultimately optimizing the amount of used sensors and their placement. The tool’s design offers significant improvements in aircraft fuel gauging accuracy, directly impacting aerostructures and instrumentation, and it is a key aspect of flight safety, fuel management, and navigation in aerospace technology.

Density gradient-RRT: An improved rapidly exploring random tree algorithm for UAV path planning

In-depth studies of algorithms for solving motion planning problems have been conducted due to the rapid popularization and development of unmanned aerial vehicles in previous decades. Among them, the classic rapidly exploring random tree (RRT) algorithm has derivative algorithms (e.g., RRT*, Q-RRT*, and F-RRT*) that focus on the optimal path cost of the initial solution. Other improved algorithms, such as RRT-connect and BG-RRT, focus on the optimal time of the initial solution. This article proposes an improved density gradient-RRT (DG-RRT) algorithm based on RRT that improves the efficiency of the guide point and reduces the time lost in the process of obtaining the initial solution through the dynamic gradient sampling strategy. Simultaneously, it reduces the path cost by reconstructing the output path. The proposed algorithm is an expansion algorithm of a random tree, and the performance of the algorithm can be further improved by combining it with other RRT optimization algorithms. DG-RRT and other algorithms are compared in different environments through simulation experiments to verify the advantages of DG-RRT. In addition, it used a set of simulation flight tests to verify the feasibility of the DG-RRT algorithm for UAV path planning.

Complexity of the instantaneous frequency variation in auditory steady-state response: A high sensitivity, high anti-interference index of mental fatigue

The quantification of mental fatigue (MF) in aviation remains a long-lasting problem which has a bearing on flight safety. In this study, we have developed a complexity-based MF index based on multiscale entropy (MSE) analysis of instantaneous frequency variation (IFV) in auditory steady-state response (ASSR). Subjects were asked to undertake a flight simulation task, before and after which, electroencephalograms (EEGs) were recorded with no auditory stimulus, and with Don chirp sound to elicit ASSR. In the anti-interference tests, our proposed MF index showed higher performance in terms of anti-interference from electrooculogram (EOG) and environment. In the flight simulation task, fatigue evaluations based on ASSR have achieved a better fatigue detection effect comparing with the conventional methods: 95% in the fatigue detection rate and −13.885% in the MF index. In conclusion, the proposed MF index has high sensitivity and high anti-interference capacities, therefore has potential applications in a variety of fields that require robust monitoring of MF outside a shielded chamber.

Tuning the Proportional-Integral-Derivative Control Parameters of Unmanned Aerial Vehicles Using Artificial Neural Networks for Point-to-Point Trajectory Approach.

Nowadays, trajectory control is a significant issue for unmanned micro aerial vehicles (MAVs) due to large disturbances such as wind and storms. Trajectory control is typically implemented using a proportional-integral-derivative (PID) controller. In order to achieve high accuracy in trajectory tracking, it is essential to set the PID gain parameters to optimum values. For this reason, separate gain values are set for roll, pitch and yaw movements before autonomous flight in quadrotor systems. Traditionally, this adjustment is performed manually or automatically in autotune mode. Given the constraints of narrow orchard corridors, the use of manual or autotune mode is neither practical nor effective, as the quadrotor system has to fly in narrow apple orchard corridors covered with hail nets. These reasons require the development of an innovative solution specific to quadrotor vehicles designed for constrained areas such as apple orchards. This paper recognizes the need for effective trajectory control in quadrotors and proposes a novel neural network-based approach to tuning the optimal PID control parameters. This new approach not only improves trajectory control efficiency but also addresses the unique challenges posed by environments with constrained locational characteristics. Flight simulations using the proposed neural network models have demonstrated successful trajectory tracking performance and highlighted the superiority of the feed-forward back propagation network (FFBPN), especially in latitude tracking within 7.52745 × 10-5 RMSE trajectory error. Simulation results support the high performance of the proposed approach for the development of automatic flight capabilities in challenging environments.

Effects of age and flight experience on prefrontal cortex activity in airline pilots: An fNIRS study

It is essential for airlines to have a deep understanding of the cognitive impact of aging among pilots. The current literature on executive function indicates that compensatory mechanisms in the brain may counteract age-related cognitive decline, at least up to certain task load levels. However, few studies have been administered to evaluate changes in aircrew competence as they age. The present study focuses on dorsolateral prefrontal cortex (DLPFC) activity as it is implicated in cognitive performance and working memory, which are associated with skill proficiency. We measured the DLPFC activity for airline pilots, including trainees, during maneuvering using a flight simulator. Our preliminary results indicated that only expert (aged) pilots demonstrated higher activity of the left DLPFC than the right one. However, for youth trainees, not only was the error rate high while using the flight simulator, but the activity of the DLFPC was also lower than that of the expert pilots, and there was no statistically significant difference between the left and right DLPFC. Although these findings partially differ from those reported in previous studies on age-related changes, it is evident that training as an airline pilot for over 20 years may affect such results. We believe that this noninvasive approach to objective quantification of skill will facilitate the development of effective assessment competence in aging.

Flight Trainee Performance Evaluation Using Gradient Boosting Decision Tree, Particle Swarm Optimization, and Convolutional Neural Network (GBDT-PSO-CNN) in Simulated Flights

Flight simulation training is one of the most important methods in early-stage civil aviation flight training. In this regard, flight simulation competitions are effective tools for evaluating the flight skills of trainees. In this study, a model is developed for evaluating the flight skills of trainees by integrating GBDT (Gradient Boosting Decision Tree), PSO (Particle Swarm Optimization), and CNNs (Convolutional Neural Networks). Flight data from simulations is employed for model training. Initially, performance data and scores are gathered from a simulated flight competition platform. The GBDT algorithm is then applied to filter and identify essential flight parameters from the collected data. Subsequently, the PSO-CNN model is utilized to train on the extracted flight parameters. The proposed GBDT-PSO-CNN model achieves a recognition rate of 93.8% on the test dataset. This assessment system is of significant importance for improving the specific maneuvering skill level of flight trainees.

Investigating Tailless UAV Flight Dynamics through Modeling, Simulation, and Flight Testing

Tailless Unmanned Aerial Vehicles (UAVs) offer several advantages over their conventional counterparts, including enhanced maneuverability, higher durability, and better stealth. However, they are challenging in their stability and control due to the absence of stabilizing and control surfaces that typically exist in conventional empennage. A crucial process for analyzing the tailless UAVs’ stability/performance characteristics and determining/validating their flight control parameters is done via the modeling and simulation of flight dynamics. This paper presents a comprehensive and systematic procedure for investigating the flight dynamics of a tailless UAV, including modeling, simulation, analytical verification, and flight testing, while also explaining the interconnections among these elements. It also addresses the common challenge of limited accessibility of UAV essential data through using diverse analytical, empirical, and experimental methods. First, a rapid and effective first-principles modeling approach is introduced to simulate the nonlinear six-degree-of-freedom flight dynamics of a small tailless UAV case study. The modeling process follows a modular framework where well-defined experiments and commercial of-the-shelf software, tools, and sensors are employed to build the necessary sub-models, including geometric, mass–inertia, aerodynamic, propulsion, and actuator models. Then, all sub-models are integrated into a simulation environment to allow the prediction of the UAV dynamic response obtained from the given control inputs. The developed flight dynamic model is subjected to a thorough verification process to ensure its integrity and proper functionality by comparing the simulated trim and natural flight modes with the calculated analytical results. Finally, a set of specific flight tests are performed to validate the developed simulation model and verify relevant performance characteristics for the case-study UAV. The results show that the proposed approach provides a systematic and straightforward method for examining the flight dynamics of small tailless UAVs with reasonable accuracy.

Autonomous flight performance maximization for slung load carrying rotary wing mini unmanned aerial vehicle

Purpose This research study aims to minimize autonomous flight cost and maximize autonomous flight performance of a slung load carrying rotary wing mini unmanned aerial vehicle (i.e. UAV) by stochastically optimizing autonomous flight control system (AFCS) parameters. For minimizing autonomous flight cost and maximizing autonomous flight performance, a stochastic design approach is benefitted over certain parameters (i.e. gains of longitudinal PID controller of a hierarchical autopilot system) meanwhile lower and upper constraints exist on these design parameters. Design/methodology/approach A rotary wing mini UAV is produced in drone Laboratory of Iskenderun Technical University. This rotary wing UAV has three blades main rotor, fuselage, landing gear and tail rotor. It is also able to carry slung loads. AFCS variables (i.e. gains of longitudinal PID controller of hierarchical autopilot system) are stochastically optimized to minimize autonomous flight cost capturing rise time, settling time and overshoot during longitudinal flight and to maximize autonomous flight performance. Found outcomes are applied during composing rotary wing mini UAV autonomous flight simulations. Findings By using stochastic optimization of AFCS for rotary wing mini UAVs carrying slung loads over previously mentioned gains longitudinal PID controller when there are lower and upper constraints on these variables, a high autonomous performance having rotary wing mini UAV is obtained. Research limitations/implications Approval of Directorate General of Civil Aviation in Republic of Türkiye is essential for real-time rotary wing mini UAV autonomous flights. Practical implications Stochastic optimization of AFCS for rotary wing mini UAVs carrying slung loads is properly valuable for recovering autonomous flight performance cost of any rotary wing mini UAV. Originality/value Establishing a novel procedure for improving autonomous flight performance cost of a rotary wing mini UAV carrying slung loads and introducing a new process performing stochastic optimization of AFCS for rotary wing mini UAVs carrying slung loads meanwhile there exists upper and lower bounds on design variables.

A Weighted Feature Fusion Model for Unsteady Aerodynamic Modeling at High Angles of Attack

Unsteady aerodynamic prediction at high angles of attack is of great importance to the design and development of advanced fighters. In this paper, a weighted feature fusion model (WFFM) that combines the state-space model and neural networks is proposed to build an unsteady aerodynamic model for the precise simulation and control of post-stall maneuvers. In the proposed model, the influences of the physical model on neural networks are considered and adjusted by introducing a standardization layer and a new weighting method. A long short-term memory (LSTM) network is used to fuse two mappings: one from flight states to aerodynamic loads, and the other from low-fidelity data to high-fidelity data. Data from wind tunnel oscillation experiments at high angles of attack using a new kind of wire-driven parallel robot and the traditional tail support are used for verifying the proposed aerodynamic model. The output of the WFFM is also compared with predictions from other models, such as the state-space model, single LSTM model, and feature fusion model not including a feature weighting layer. Results demonstrate improved accuracy of the proposed model in the interpolation and extrapolation tests. Furthermore, the WFFM is applied to the flight simulation of F-16 with different control inputs. Compared with conventional models, the WFFM shows improved accuracy and better generalization capability.

Formal modelling and verification of autonomous reasoning based flight simulation system

The emerging trends towards intelligent computing have drastically changed the human lifestyle in every facet of their lives. Due to the swift escalation of smart systems and emerging technologies, human life has become much more dependent and even more addicted to fulfilling their desire using smart and tiny resource-bounded gadgets. This revolution has evolved towards autonomous decision support systems. Autonomous decision support systems have the ability to acquire information autonomously, reason the information, and adapt behavior accordingly. As these systems are deployed in a highly decentralized environment and exhibit complex adaptive behavior, however, the inconsistent nature of information may raise different challenging issues. This paper presents a multi-agent environmentally-aware framework for modeling and reasoning flight management systems. This system has a sound reasoning mechanism to execute and monitor flight control activities while considering liveness and safety-critical constraints. We use the UPPAAL model checker to formally analyze the system’s behavior and verify correctness properties.

Multimodal decoding of error processing in a virtual reality flight simulation

Technological advances in head-mounted displays (HMDs) facilitate the acquisition of physiological data of the user, such as gaze, pupil size, or heart rate. Still, interactions with such systems can be prone to errors, including unintended behavior or unexpected changes in the presented virtual environments. In this study, we investigated if multimodal physiological data can be used to decode error processing, which has been studied, to date, with brain signals only. We examined the feasibility of decoding errors solely with pupil size data and proposed a hybrid decoding approach combining electroencephalographic (EEG) and pupillometric signals. Moreover, we analyzed if hybrid approaches can improve existing EEG-based classification approaches and focused on setups that offer increased usability for practical applications, such as the presented game-like virtual reality flight simulation. Our results indicate that classifiers trained with pupil size data can decode errors above chance. Moreover, hybrid approaches yielded improved performance compared to EEG-based decoders in setups with a reduced number of channels, which is crucial for many out-of-the-lab scenarios. These findings contribute to the development of hybrid brain-computer interfaces, particularly in combination with wearable devices, which allow for easy acquisition of additional physiological data.